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OBJECTIVE: 

Background 

In  the  21st  century  Navy,  war-fighting  objectives  include  technological  superiority,  reduced  staffing,  and  an 
affordable  force.  Given  the  requirement  of  superior  performance  with  fewer  personnel,  the  Navy  has  a  critical  need 
for  research  that  addresses  selection  and  classification  of  sailors  into  billets.  The  present  research  investigated  the 
relationship  between  individual  differences  in  cognitive  and  noncognitive  abilities  and  performance  in  interactive 
task  environments  at  both  individual  and  team  levels.  Because  this  research  spans  multiple  disciplines,  our  research 
team  was  structured  accordingly.  Our  collaborative  team  included  electrical  and  computer  engineers,  computer 
scientists,  cognitive  scientists  and  cognitive  psychologists. 

Project  Goals 

The  specific  goal  of  this  project  was  to  investigate  the  impact  of  individual  differences  in  cognitive  and 
noncognitive  abilities  on  team  performance.  Cognitive  and  social  psychology  measures  were  used  to  assess 
individual  differences  in  cognitive  and  noncognitive  abilities  and  relate  them  to  a)  individual  performance  and  b) 
team  task  performance.  This  research  will  be  useful  for  expanding  measures  used  to  select  sailors  to  serve  in  the  2f  * 
Century  Navy.  This  research  will  address  issues  important  to  sailor  selection  and  classification  into  billets  that 
require  team-intensive  skills,  including  structuring  teams  according  to  team-member  abilities. 

A  vast  literature  exists  on  leadership  and  team  abilities,  although  much  is  considered  to  be  anecdotal  or  theoretical 
in  nature  (Bartone,  Snook,  &  Tremble,  2002;  Brannick,  Roach,  &  Salas,  1993;  Druckman,  Singer,  &  Van  Cott, 
1997,  p.  98;  LePine,  2003;  Stout,  Salas,  &  Fowlkes,  1997).  There  have  been  recent  studies  that  provide  support  for 
the  role  of  cognitive  abilities,  such  as  problem  solving  (e.g.,  Phillips  &  Hunt,  1992;  Fiore,  Cuevas,  Scielzo,  & 
Salas,  2002),  and  noncognitive  factors,  such  as  personality  traits  (e.g.,  Bass,  1990;  Mohammed,  &  Angell,  2003; 
Porter,  Hollenbeck,  Ilgen,  Ellis,  West,  &  Moon,  2003),  in  leadership  and  team-related  performance.  However,  to 
our  knowledge  there  are  no  published  studies  that  examine  the  role  of  individual  differences  in  cognitive  and 
noncognitive  abilities  in  team  performance  in  a  dynamic,  interactive  team  task  environment.  There  are  studies  that 
use  what  are  termed  “microworlds”,  where  an  individual  subject  may  control  a  team  of  computer  agents,  but  the 
subject  is  not  working  as  a  team  member  per  se  (e.g.,  Rigas,  Carling,  &  Brehmer,  2002;  but  see  Cooke,  Kiekel,  & 
Helm,  2001). 

In  the  military  operational  environment,  sailors  serve  as  members  of  a  team  charged  with  working  toward  a  common 
goal.  The  operational  environment  is  dynamic,  where  the  state  of  world  can  change  quickly,  and  the  team  must 
adapt  to  the  changes  to  achieve  their  mission.  There  is  usually  a  hierarchical  structure  to  the  team,  where  an 
operational  supervisor  (e.g.,  a  commander)  is  responsible  for  the  success  of  the  mission,  but  relies  upon  multiple 
personnel  to  execute  the  requisite  mission  tasks.  For  operational  teams  to  accomplish  their  mission,  they  must 
develop  a  shared  understanding  of  their  operational  environment,  communicate  that  shared  understanding,  and  do  so 
in  a  timely  fashion  to  keep  pace  with  the  dynamic  changes  caused  by  actions  of  the  enemy,  environmental 
influences,  and  so  on. 

In  order  to  achieve  our  goal,  we  sought  to  emulate  the  characteristics  of  this  type  of  operational  environment.  That 
is,  our  tasks  took  place  in  dynamically  changing  environments  in  which  team  members  were  often  required  to 
coordinate  their  actions  in  order  to  achieve  optimal  team  performance.  Our  goal  was  achieved  by  relating 
performance  metrics  on  these  tasks  to  measures  of  individual  differences  in  cognitive  and  noncognitive  abilities. 
Measures  of  verbal  comprehension,  spatial  orientation  and  visualization,  and  general  reasoning  ability  served  as 
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cognitive  ability  measures.  Measures  of  social  intelligence  and  personality  (e.g.,  five-factor  personality  inventory) 
factors  served  as  noncognitive  measures. 

In  the  current  research,  team  performance  was  measured  in  a  dynamic  and  interactive  task  environment  that  served  as 
an  analogue  to  the  military  operational  environment  just  described.  Specifically,  participants  served  as  team 
members  in  a  computer  game  called  “Half-Life”  (Le,  1998).  For  the  current  research,  a  set  of  twelve  tasks  was 
developed  in  the  computerized  Half-Life  environment.  These  twelve  tasks  were  designed  to  provide  measurement 
opportunities  for  multiple  dimensions  of  teamwork  (Dickinson  &  McIntyre,  1997).  In  each  task,  the  participants 
were  required  to  escort  a  preprogrammed  VIP  (i.e.,  VIP  movements  were  predetermined  through  the  computer 
program)  to  a  target  destination.  In  addition  to  the  human  team  members  and  the  computerized  VIP  agent,  the 
environments  contained  two  other  types  of  computer-controlled  agents.  Hostile  agents  would  fire  at  the  VIP  or  the 
team  members.  Neutral  agents  would  walk  in  Ae  environment  but  not  pose  a  threat  to  the  VIP.  Team  members 
were  instructed  not  to  damage  neutral  agents.  See  Figure  1 '  for  an  example  illustration  of  the  environment  with 
specific  elements  described  in  greater  detail. 

APPROACH: 

In  order  to  assess  the  role  of  individual  differences  in  individual  and  team  performance,  several  factors  had  to  be 
considered  during  our  research.  First,  a  set  of  ability  tests  needed  to  be  chosen  in  order  to  capture  the  relevant 
attributes  of  team  members  that  might  be  useful  in  predicting  task  performance.  Second,  given  the  results  of  the 
ability  tests,  an  experimental  design  had  to  be  constructed  that  would  allow  us  to  study  the  relative  impact  of 
cognitive  and  noncognitive  abilities  on  task  performance.  Finally,  a  set  of  tasks  had  to  be  constructed  in  a  manner 
that  would  both  reflect  the  attributes  of  the  operational  environments  relevant  to  the  Navy  and  provide  ample 
opportunities  to  measure  performance  in  a  systematic  way.  Each  of  these  topics  will  be  discussed  in  turn. 

Measuring  Individual  Differences 

In  order  to  measure  individual  differences  in  cognitive  and  noncognitive  abilities,  all  participants  completed  a 
battery  of  tests.  This  battery  of  tests  was  selected  following  discussions  with  personnel  at  NPRST.  The  test 
battery  included  four  cognitive  ability  tests  that  measured  verbal,  spatial,  and  general  reasoning  abilities. 
Specifically,  the  four  tests  chosen  were:  Raven’s  Standard  Progressive  Matrices  (Raven,  1938),  Integrating  Details 
(Alderton,  1989),  Thurstone’s  Perceptual  Speed  (Thurstone,  1983),  and  Nelson-Denny  Vocabulary  (Brown,  Bennett, 
&  Hanna,  1981).  Figures  2  through  5  depict  screenshots  of  the  cognitive  ability  tests  used  in  our  study.  The 
battery  also  included  three  noncognitive  ability  tests  that  measured  various  aspects  of  personality  and  social 
abilities.  Specifically,  the  three  tests  were  the  International  Personality  Item  Pool  (IPIP;  Goldberg,  1999),  the 
Enlisted  Navy  Computer  Adaptive  Personality  Scales  (ENCAPS;  Farmer  &  Bearden,  2004),  and  the  Positive  and 
Negative  Affect  Scale  (PANAS;  Watson,  Clark,  &  Tellegen,  1988).  The  seven-test  battery  was  administered  in 
approximately  two  hours  over  a  two-day  period.  Following  completion  of  the  test  battery,  we  performed  a  principal 
components  factor  analysis  on  the  cognitive  battery  results  to  determine  if  a  subset  of  the  scores  could  be  grouped 
together  into  a  cognitive  factor  ability.  The  results  showed  that  the  four  cognitive  tests  loaded  significantly  on  a 
single  factor.  We  therefore  calculated  a  single  cognitive  factor  score  for  each  participant  and  used  that  score  in 
defining  our  groups  (discussed  below).  Similar  analyses  of  the  noncognitive  scores  resulted  in  multiple  factors. 
Because  of  the  limitations  in  the  size  of  and  funds  for  a  participant  pool,  we  needed  to  select  one  noncognitive 
factor  for  noncognitive  grouping.  To  do  this,  we  consulted  with  subject-matter-experts  (SMEs)  from  NPRST 
including  Drs.  David  Alderton,  William  Farmer,  Tanja  Blackstone,  and  Jacqueline  Mottem.  The  SMEs 
recommended  using  Social  Orientation  from  the  ENCAPS  test  as  the  noncognitive  measure  of  interest  for  dividing 
participants  into  groups. 

Experimental  Design  and  Tasks 

One  hundred  forty-seven  participants  were  recruited  to  take  our  battery  of  cognitive  and  noncognitive  ability  tests. 
The  147  participants  were  later  divided  into  teams  based  on  the  results  of  the  test  battery  and  the  need  for  a  balanced 
design.  Our  final  experimental  design  called  for  four  groups  of  participants.  The  first  group  of  participants  (HH) 
was  high  in  cognitive  ability  and  high  in  noncognitive  ability  (i.e..  Social  Orientation).  The  second  group  of 
participants  (HL)  was  high  in  cognitive  ability,  but  low  in  noncognitive  ability.  The  third  group  of  participants 
(LH)  was  low  in  cognitive  ability,  but  high  in  noncognitive  ability.  The  fourth  group  of  participants  (LL)  was  low 
in  cognitive  ability  and  low  in  noncognitive  ability.  Recall  that  the  cognitive  test  scores  loaded  on  a  single  factor. 
That  factor  score  defined  cognitive  grouping.  Scores  on  the  Social  Orientation  scale  from  ENCAPS  defined 
noncognitive  grouping. 


^  All  Figures  and  Tables  are  included  in  the  Appendix 
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Once  classified  into  one  of  the  four  possible  groups,  participants  were  placed  in  teams  that  would  work  together  to 
complete  the  experimental  tasks.  Each  team  was  composed  of  three  members;  one  leader  and  two  subordinates. 

The  team  leaders  were  always  from  the  HH  group.  Team  subordinates  were  from  one  of  the  four  groups.  Within 
each  team,  subordinate  ability  was  held  constant.  That  is,  they  were  either  both  HH,  both  HL,  both  LH,  or  both 
LL.  Table  1  shows  the  four  team  types  and  the  number  of  each  type  of  team  that  completed  the  entire  experiment. 

Teams  were  run  through  a  series  of  tasks  developed  in  the  “Half-Life”  computer  game.  As  previously  mentioned, 
the  teams  were  assigned  to  escort  a  computer-controlled  VIP  to  a  target  destination  while  protecting  him  from 
hostile  agents.  In  addition  to  that  general  goal,  each  of  the  twelve  tasks  took  place  in  a  different  computer  generated 
“environment”  that  imposed  a  unique  challenge.  For  example,  in  one  of  the  twelve  tasks,  the  VIP  would  walk 
through  the  environment  until  encountering  an  obstacle  that  blocked  his  path  (see  Figures  6a  —  6e).  The  team 
members  had  to  split  up  to  achieve  both  goals.  That  is,  they  had  to  disperse  in  order  to  both  protect  the  VIP  and 
find  a  button  in  the  environment  that  would  remove  the  obstacle  and  let  them  proceed  to  the  target  destination.  See 
Figures  6  and  7  for  example  illustrations  of  what  the  tasks  looked  like  to  the  team  members  and  examples  of  other 
challenges  the  teams  had  to  overcome. 

Each  task  provided  multiple  opportunities  to  measure  individual  and  team  performance.  We  collected  significantly 
more  data  than  we  are  able  to  analyze  in  a  one-year  project.  Our  analysis  included  an  important  subset  of  the 
measures,  and  the  measures  analyzed  fell  into  one  of  three  categories  or  sets.  The  first  set  measured  individual 
performance.  The  measures  included  “individual  damage  per  engagement,”  “individual  total  player  damage,” 
“individual  shots  per  kill,”  “individual  time  per  kill,”  and  “individual  percent  of  ammunition  used  on  hostiles.”  A 
more  detailed  description  of  each  individual  performance  measure  is  provided  in  Table  2.  The  next  set  included 
composites  of  the  individual  measures.  That  is,  we  calculated  a  team  level  score  for  each  of  the  individual  level 
measures.  For  example,  each  team  member  had  three  “individual  damage  per  engagement”  scores,  but  the  scores 
were  averaged  across  the  three  team  members  to  calculate  a  “team  composite  damage  per  engagement”  score.  See 
Table  3  for  a  description  of  composite  team  measures.  All  of  these  composite  measures  are  a  function  of  the 
individual  measures,  but  are  influenced  by  how  well  the  team  coordinates  their  actions.  This  is  particularly  the  case 
for  “team  composite  damage  per  engagement”  and  “team  composite  total  player  damage.”  Coordinated  actions  in 
combat  lead  to  fewer  opportunities  for  enemy  agents  to  score  hits.  So,  a  well-coordinated  team  would  have  a  lower 
damage  score.  Finally,  the  third  set  of  measures  represent  purely  team  level  scores  that  are  not  a  function  of 
individual  member  performance.  The  measures  include  “team  time  to  completion,”  “team  VIP  damage,”  “team 
percent  time  VIP  guarded,”  and  “team  percent  time  VIP  in  danger.”  For  a  more  detailed  description  of  each  of  the 
team  measures,  see  Table  4. 

ACCOMPLISHMENTS  (throughout  award  period): 

We  accomplished  our  goal  of  studying  the  impact  of  cognitive  and  noncognitive  abilities  on  individual  and  team 
performance.  To  accomplish  this  goal,  we  administered  a  set  of  ability  tests,  constructed  specific  sets  of  teams,  and 
measured  performance  on  a  carefully  constructed  set  of  tasks.  Our  analyses  of  the  performance  measures  at  the 
individual  and  team  levels  will  be  discussed  in  turn  below.  In  addition  to  fulfilling  this  main  goal  of  the  research, 
we  constructed  an  infrastructure  to  support  this  effort  and  potential  future  efforts.  The  specific  tools  we  have 
developed  as  part  of  our  infrastructure  are  discussed  below  as  well. 

Performance  Measure  Analyses 

The  individual,  team  composite,  and  team  measures  described  above,  were  used  to  derive  individual  and  team  level 
factor  scores  using  a  principal  components  factor  analysis  with  Promax  rotation.  The  team  level  analysis  included 
the  team  composite  measures  and  the  team  measures.  Tables  5  and  6  show  the  results  of  these  analyses  in  detail. 
Two  individual  measure  factors  accounted  for  70%  of  the  variance  in  the  data.  We  labeled  the  factors  “combat 
effectiveness,”  and  “shooting  efficiency”  (see  Table  5).  At  the  team  level,  three  factors  accounted  for  68%  of  the 
variance  in  the  data.  We  labeled  these  factors  “coordinative  ability,”  “VIP  protection,”  and  “shooting  type”  (see 
Table  6).  After  calculating  individual  and  team  level  scores  for  each  factor,  our  next  step  was  to  relate  those  factor 
scores  to  the  cognitive  and  noncognitive  abilities  previously  measured  for  each  participant. 

Individual  Performance  Analyses 

Our  goal  was  to  relate  individual  measure  factor  scores  to  individual  difference  measures.  We  performed  two 
regression  analyses  using  the  factor  scores  as  dependant  variables  to  be  predicted  by  a  set  of  independent  variables 
consisting  of  either  the  individual  cognitive  and  IPIP  scores,  or  the  individual  cognitive  and  ENCAPS  scores. 
Tables  7  and  8  summarize  the  significant  results  from  these  analyses.  To  briefly  summarize,  results  from  the  first 
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regression  indicate  that  cognitive  ability  and  scores  on  the  IPIP  (especially  Neuroticism)  predicted  “combat 
effectiveness,”  accounting  for  2 1  %  of  the  variance  in  the  data.  That  is,  higher  cognitive  ability  and  higher 
emotional  stability  yielded  better  factor  scores  on  “combat  effectiveness,”  a  factor  that  measured  the  average  amount 
of  damage  taken  in  each  engagement  and  each  task  as  a  whole.  Additionally,  results  from  the  second  regression 
indicate  scores  on  the  ENCAPS  test  (especially  Social  Orientation)  predicted  “shooting  efficiency,”  accounting  for 
7%  of  the  variance  in  the  data.  That  is,  higher  Social  Orientation  scores  yielded  better  scores  on  “shooting 
efficiency,”  meaning  individuals  took  fewer  shots  and  less  time  to  eliminate  hostile  agents,  but  tended  to  waste 
more  ammunition.  One  possible  reason  for  this  relationship  might  be  that  individuals  with  higher  Social 
Orientation  are  more  likely  to  respond  to  feedback  from  other  team  members  about  efficient  methods  of  eliminating 
hostiles  (e.g.,  aiming  for  critical  targets  like  the  head).  Overall,  these  analyses  suggest  that  individual  performance 
metrics  on  a  dynamic,  team  task  can  be  predicted  by  measures  of  cognitive  and  noncognitive  abilities. 

Team  Performance  Analyses 

While  individual  level  performance  is  important,  such  analyses  do  not  address  the  effects  of  team  composition  on 
performance.  To  fully  explore  the  effects  of  team  composition  on  team  performance,  we  used  multiple  analysis 
approaches.  The  team  composite  and  team  measures  were  both  used  in  the  following  analyses.  Recall  that  team 
level  variables  are  independent  of  individual  level  performance  measures  (see  Table  4),  while  composite  team 
measures  were  calculated  by  averaging  individual  performance  measures  (see  Table  3). 

Team  Composition 

We  examined  the  effect  of  team  composition  (as  defined  in  our  experimental  design,  see  Figure  8)  on  team 
composite  and  team  measures  of  performance.  Recall  that  these  measures  loaded  significantly  on  three  factors  in  a 
principal  components  factor  analysis  (i.e.,  “coordinative  ability,”  “VIP  protection,”  and  “shooting  type;”  see  Table 
6).  An  analysis  of  variance  was  performed  with  team  composition  (i.e.,  high  cognitive,  high  noncognitive;  high 
cognitive,  low  noncognitive;  low  cognitive,  high  noncognitive;  or  low  cognitive,  low  noncognitive)  as  a  between 
subjects  variable,  and  each  of  the  three  team  performance  factors  as  within  subjects  variables.  There  was  a 
significant  effect  of  team  composition  on  “coordinative  ability,”  F  (3,  40)  =  4.15,/?  <  .02.  Further,  we  performed  a 
separate  2x2  analysis  of  variance  to  determine  if  one  component  of  the  team  composition  (i.e.,  cognitive  or 
noncognitive)  or  the  interaction  of  the  two  was  contributing  most  to  the  effect.  We  used  team  cognitive 
composition  (i.e.,  high  or  low)  and  team  noncognitive  composition  (high  or  low)  as  between  subjects  variables,  and 
the  “coordinative  ability”  factor  scores  as  within  subjects  variables.  There  was  a  main  effect  of  the  cognitive 
component,  F  (1,  40)  -  10.93,/?  <  .01,  but  no  significant  effect  of  the  noncognitive  component  or  the  interaction. 
This  implies  that  cognitive  ability  and  not  Social  Orientation  is  the  primary  factor  in  a  team’s  “coordinative 
ability.”  No  significant  effects  for  team  composition  (as  defined  in  our  experimental  design)  were  found  for  the 
other  two  dependant  variable  factors. 

Team  Composition  Models 

In  addition  to  analyzing  team  performance  as  a  function  of  our  experimentally  controlled  team  compositions,  we 
evaluated  a  set  of  potential  team  performance  ‘models’  to  explore  the  dynamics  of  team  performance  in  more  detail. 
The  goal  of  using  these  models  was  to  understand  more  about  how  the  structure  of  member  cognitive  and 
noncognitive  scores  contributed  to  team  level  performance.  The  models  we  used  were  called  “compensatory,” 
“synergy,”  “weakest-link,”  and  “average”  models;  and  each  is  discussed  below.  For  each  model,  we  performed  two 
sets  of  regression  analyses.  The  first  set  used  cognitive  ability  and  IPIP  scores  as  predictors,  while  the  second  set 
used  cognitive  ability  and  ENCAPS  scores  as  predictors.  These  analyses  allowed  us  to  determine  the  contributions 
of  noncognitive  abilities  measured  by  the  IPIP  scales  separately  from  the  contribution  of  noncognitive  abilities 
measured  by  the  ENCAPS  scales. 

The  compensatory  model  evaluates  the  hypothesis  that  the  team’s  performance  can  be  best  predicted  as  a  function  of 
the  best  scores  (i.e.,  cognitive  and  noncognitive  ability  scores)  represented  on  the  team.  To  test  the  compensatory 
model,  we  found  the  highest  score  for  cognitive  and  for  each  noncognitive  ability  on  the  team  and  related  those 
scores  to  the  team  performance  measures.  The  compensatory  model  did  not  significantly  predict  any  of  the  team 
factor  scores,  thus  we  conclude  that  the  team’s  strongest  member  is  not  the  primary  contributing  factor  to  team 
performance. 

Next,  the  synergy  model  evaluates  the  hypothesis  that  the  team’s  performance  can  be  best  predicted  by  the 
interaction  of  the  individual  team  member  scores.  To  test  the  synergy  model,  we  calculated  a  product  term  for  each 
team’s  cognitive  and  noncognitive  abilities  and  related  those  product  terms  to  the  team  performance  measures.  The 
significant  results  of  the  synergy  model  are  shown  in  Tables  9a  and  9b.  The  cognitive  and  IPIP  scores  in  the 
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synergy  model  significantly  predicted  “VIP  protection,”  accounting  for  17%  of  the  variance  in  the  data.  The 
cognitive  and  ENCAPS  scores  in  the  synergy  model  significantly  predicted  “shooting  type,”  accounting  for  20%  of 
the  variance  in  the  data. 

The  third  model,  “weakest-link,”  evaluates  the  hypothesis  that  the  team’s  performance  can  be  best  predicted  as  a 
function  of  the  lowest  scores  represented  on  the  team.  To  test  the  weakest-link  model,  we  found  the  lowest  score 
for  cognitive  and  for  each  noncognitive  ability  on  the  team  and  related  those  scores  to  the  team  performance 
measures.  The  significant  results  of  the  weakest-link  model  are  shown  in  Tables  10a  through  lOd.  The  cognitive 
and  IPIP  scores  in  the  weakest-link  model  significantly  predicted  “coordinative  ability,”  accounting  for  36%  of  the 
variance  in  the  data.  The  cognitive  and  IPIP  scores  in  the  weakest-link  model  also  predicted  “VIP  protection,” 
accounting  for  32%  of  the  variance  in  the  data.  The  cognitive  and  ENCAPS  scores  in  the  weakest-link  model 
predicted  “shooting  type,”  accounting  for  23%  of  the  variance  in  the  data. 

Finally,  the  average  model  evaluates  the  hypothesis  that  the  team’s  performance  can  be  best  predicted  as  a  fimction 
of  the  mean  scores  represented  on  the  team.  To  test  the  average  model,  we  calculated  the  mean  score  for  cognitive 
and  for  each  noncognitive  ability  on  the  team  and  related  those  scores  to  the  team  performance  measures.  The 
significant  results  of  the  average  model  are  shown  in  Tables  11a  through  lid.  The  cognitive  and  IPIP  scores  in  the 
average  model  significantly  predicted  “coordinative  ability,”  accounting  for  27%  of  the  variance  in  the  data.  The 
cognitive  and  IPIP  scores  in  the  average  model  marginally  predicted  “VIP  protection,”  accounting  for  15%  of  the 
variance  in  the  data.  The  cognitive  and  ENCAPS  scores  in  the  average  model  predicted  “shooting  type,”  accounting 
for  14%  of  the  variance  in  the  data. 

Taken  together,  the  results  of  these  analyses  show  that  the  weakest-link  model  accounts  for  the  most  variance  in  the 
data,  consistent  with  findings  obtained  in  previous  research  (e.g.,  LePine,  Hollenbeck,  &  Ilgen,  1997).  This 
finding  is  important  to  consider  when  structuring  teams.  One  possible  conclusion  firom  these  analyses  is  that 
weaker  members  should  not  be  included  in  teams  that  are  assigned  to  highly  critical  team  tasks.  This  is  because  the 
weakest  member  appears  to  dictate  overall  team  performance. 

Individual  Differences  and  Team  Performance 

A  final  set  of  analyses  we  performed  examined  the  relationship  between  cognitive  and  noncognitive  abilities  and 
team  performance  regardless  of  team  composition.  That  is,  we  used  the  set  of  ability  scores  from  each  team  member 
together  in  a  set  of  regression  analyses  to  predict  the  team  factor  scores  as  dependant  variables.  This  is  essentially  a 
‘many-to-one’  approach  as  illustrated  in  Figure  9.  The  results  of  these  analyses  are  shown  in  Tables  12  through  14. 
The  findings  show  that  cognitive  ability  and  IPIP  scores  (particularly  Neuroticism  and  Extraversion)  predict 
“coordinative  ability,”  accounting  for  40%  of  the  variance  in  the  data.  That  is,  teams  with  higher  cognitive  ability, 
higher  emotional  stability,  and  less  extraverted  team  leaders  had  better  scores  on  “coordinative  ability,”  taking  less 
time  to  complete  tasks,  and  less  damage  during  the  tasks.  IPIP  scores  (Neuroticism  and  Extraversion)  also 
predicted  “VIP  protection,”  accounting  for  35%  of  the  variance  in  the  data.  That  is,  teams  with  more  emotional 
stability  and  less  extraverted  team  leaders  had  better  scores  on  “VIP  protection,”  leaving  the  VIP  unguarded  less  of 
the  time,  and  keeping  the  VIP  from  taking  as  much  damage  during  the  tasks.  The  third  dependant  factor,  “shooting 
type”  was  predicted  by  scores  on  the  ENCAPS  test,  accounting  for  33%  of  the  variance  in  the  data.  That  is,  teams 
with  higher  Achievement  Motivation,  higher  Social  Orientation,  and  higher  Stress  Tolerance  had  better  scores  on 
the  “shooting  type”  factor,  taking  less  shots  and  less  time  to  eliminate  hostiles;  but  also  tending  to  waste  a  larger 
percentage  of  their  ammunition.  These  analyses  further  support  the  importance  of  using  cognitive  and  noncognitive 
measures  in  selection  and  classification  because  the  impact  of  individual  differences  can  be  seen  both  with  and 
without  regard  to  team  composition. 

Software  Tools 

Evaluating  team  performance  in  a  dynamic  task  is  a  challenging  undertaking  that  requires  the  consideration  of  a 
number  of  factors.  One  factor  is  that  teams  function  in  real  time  as  the  environment  and  situations  change  around 
them.  Another  factor  is  that  a  given  behavior  might  have  different  consequences  in  different  situations  and  as  a 
result,  one  must  always  be  aware  of  context  when  evaluating  team  actions.  In  order  to  overcome  these  issues,  we 
developed  a  number  of  software  tools  that  we  used  during  this  research  and  that  could  be  easily  transitioned  to 
support  future  research  in  our  lab  or  shared  with  other  labs.  The  first  tool  we  developed  is  a  data-logging  tool  that 
is  able  to  record  specific  events  in  the  computer-based  tasks.  For  our  current  research,  this  tool  was  used  to  record 
many  variables  such  as  player  and  hostile  positions,  when  shots  are  fired,  etc.  See  Table  15  for  a  more  complete 
list  of  the  variables  this  tool  can  track.  The  second  tool  we  developed  is  a  data  extraction  tool  that  applies  specific 
rules  to  organize  the  data  output  by  the  data-logging  tool.  Specifically,  the  data  extraction  tool  extracts  the  raw  data 
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and  then  calculates  individual  and  team  level  scores.  The  data  extraction  tool  counts  the  number  of  times  a  specific 
event  occurred,  and  takes  into  account  the  contexts  (specifics)  of  the  situation  at  the  time  of  the  action.  For 
example,  if  a  participant  shot  at  a  neutral  agent  in  the  environment  during  a  massive  fire-fight  with  hostile  agents, 
one  might  choose  to  score  that  action  differently  than  if  the  player  had  shot  a  neutral  agent  during  an  otherwise  calm 
point  in  the  task.  Thus,  the  data  extraction  tool  was  designed  to  evaluate  situations  and  apply  appropriate  scoring 
rules  for  those  situations.  The  final  tool  we  developed  is  a  playback  tool  that  visually  displays  the  locations  of  the 
participants  and  agents  in  the  environment  as  they  moved  in  real-time  through  the  environment  during  the 
experiment.  The  playback  tool  is  designed  to  show  a  ‘top-down’  or  ‘bird’s-eye’  view  of  the  environment  in  order 
for  viewers  to  see  how  the  team  coordinated  their  movements  during  the  tasks.  Each  of  these  tools  was  developed 
using  C++  or  Visual  C++,  and  is  therefore  easily  portable  to  other  platforms. 

CONCLUSIONS: 

Cognitive  and  social  abilities  predict  significant  aspects  of  individual  performance  on  a  team  and  team  performance 
as  a  whole.  This  is  a  substantial  finding  from  this  one-year  effort  to  study  the  relationship  between  cognitive  and 
noncognitive  abilities  and  performance  at  the  individual  and  team  levels.  There  are  two  major  categories  of 
deliverables  associated  with  this  project.  The  first  is  the  development  of  an  infrastructure  and  tools  to  support  team 
performance  research.  The  second  is  the  experimental  collection  and  analysis  of  data  that  was  generated  fi’om  our 
tasks. 

Infrastructure 

During  the  past  year,  we  have  developed  an  infrastructure  for  conducting  team  performance  research.  This 
infrastructure  includes  a  rich  set  of  tasks  in  which  teams  can  navigate,  carry  out  simulated  battles,  and  interact  with 
elements  in  the  environment  such  as  doors,  buttons,  boxes,  etc.  In  addition  to  the  tasks,  our  infrastructure  includes 
a  set  of  tools  capable  of  collecting  real-time  performance  data,  scoring  that  data,  and  playing  back  collected  data 
such  as  team  member  movements  over  the  course  of  a  task. 

In  addition  to  our  rich  set  of  tasks,  an  experimental  methodology  was  developed  that  has  produced  findings 
consistent  with  previous  real-world  research  (e.g.,  LePine,  et  al.,  1997).  While  other  research  has  examined  certain 
aspects  of  team  performance  (e.g.,  Cooke,  et  al.,  2001),  we  know  of  no  other  research  to  use  tasks  as  closely 
analogous  to  an  operational  environment.  Because  of  the  tools  that  we  have  developed,  our  methodology  could  be 
expanded  to  investigate  specific  types  of  military  scenarios,  or  could  be  converted  to  studying  team  tasks  in  other 
domains. 

Experimental  Data 

Our  efforts  in  studying  team  performance  have  allowed  us  to  determine  that  a  relationship  exists  between  cognitive 
and  noncognitive  abilities  and  individual  performance  on  dynamic  tasks.  Recall  that  we  found  a  significant 
relationship  between  various  ability  measures  (i.e.,  cognitive  ability,  IPIP  scores,  and  ENCAPS  scores)  and  our  two 
dependant  variable  factors  reflecting  individual  performance  (i.e.,  “combat  effectiveness,”  and  “shooting  efficiency”). 
The  implication  of  these  findings  is  that  measures  of  both  cognitive  and  noncognitive  abilities  will  be  useful  in 
determining  how  well  an  individual  will  perform  on  a  set  of  dynamic  tasks.  In  addition,  our  results  showed  a 
relationship  between  cognitive  and  noncognitive  abilities  and  team  performance.  Of  particular  interest  in  our  team 
performance  level  results  is  that  the  weakest  member  of  a  team  has  a  significant  impact  on  overall  team  performance. 
One  recommendation  that  comes  from  this  finding  is  that  weaker  members  should  not  be  included  in  teams  that  are 
assigned  to  highly  critical  tasks.  This  is  because  the  weakest  member  appears  to  dictate  overall  team  performance. 

In  the  current  context  of  operational  environments,  our  research  effort  has  produced  strong  evidence  that  individual 
and  team  performance  can  be  predicted  by  measures  of  cognitive  and  noncognitive  abilities.  Including  such 
measures  in  the  Navy’s  selection  and  classification  process  would  be  of  great  benefit  and  could  bring  the  Navy 
closer  to  achieving  its  goal  of  having  a  superior  force  with  fewer  personnel.  As  a  final  note,  we  have  collected  a 
very  rich  data  set  of  which  we  have  only  analyzed  a  subset.  We  plan  to  continue  our  analysis  of  these  data  and 
publish  our  future  findings  in  relevant  scientific  journals. 

SIGNIFICANCE: 

Given  the  objective  of  a  superior  force  with  fewer  personnel,  it  is  critical  that  the  Navy  ensures  its  selection  and 
classification  of  sailors  is  performed  in  the  most  beneficial  manner  possible.  The  current  research  has  shown  that 
both  cognitive  and  noncognitive  factors  can  affect  individual  as  well  as  team  level  performance.  One  significant 
recommendation  that  comes  from  this  research  is  to  include  both  cognitive  and  noncognitive  measures  in  the 
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selection  and  classification  process.  Further,  it  may  be  useful  to  look  at  the  profile  of  sailors  based  on  these 
cognitive  and  noncognitive  measures  before  assigning  them  to  team-intensive  billets. 
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Figure  L  Example  task  environment  in  the  “Half-Life”  game.  Each  participant  views  the  environment  from  the 
first-person  perspective  and  is  equipped  with  a  handgun  (bottom  center)  and  a  radar  (top  left)  that  detects  the 
presence  of  friendly  (green  circle),  neutral  (blue  square),  and  hostile  (red  X)  agents  in  the  area.  Agents  that  are  out  of 
range  appear  as  question  marks  on  the  radar  until  the  team  member  moves  closer  to  them.  Four  types  of  characters 
exist  in  the  environment.  The  first  are  the  three  human  team  members  (e.g.,  the  green  character  on  the  far  right) 
who  can  see  each  other  in  the  environment  and  coordinate  their  actions.  The  second  type  of  character  is  the  VIP 
agent  (computerized  character;  wearing  the  white  coat,  near  the  center  of  the  image)  who  the  participants  are  assigned 
to  protect  from  the  hostile  agents.  Next,  there  are  neutral  agents  in  the  environment  (holding  the  briefcase  in  the  left 
side  of  the  image)  who  the  participants  are  instructed  not  to  injure.  Finally,  there  are  hostile  agents  (wearing  the 
body  armor  and  helmets)  who  will  fire  at  the  team  members  and  VIP. 
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Raven's  Standard  Progressive  Matrices 


Trial  1  of  ()U 


Timed  :  No 


-I-I-I 

tW 
—  1  — 1  — 1- 

i-i-i- 

1  — 1  — 1- 

oA  oB  cC 


Figure  2.  Example  trial  from  Raven’s  Standard  Progressive  Matrices. 
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Figure  3.  Example  trial  from  Thurstone’s  Perceptual  Speed. 
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Press  's'  if  same  or  Press if  different 


Figure  4.  (a)  Depicts  the  first  half  of  an  example  trial  and  (b)  depicts  the  second  half  of  an  example  trial  from 
Alderton’s  Integrating  Details. 
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Figure  6a.  In  one  task,  the  participants  are  escorting  the  male  VIP  towards  a  target  destination  when  he  encounters 
and  obstacle  and  stops.  The  team  members  must  find  a  way  to  remove  the  obstacle  in  order  to  proceed  towards  the 
destination.  Since  the  VIP  is  potentially  in  danger,  the  team  members  must  divide  their  resources  to  ensure  that  the 
VIP  is  protected  and  that  someone  can  search  through  the  environment  to  find  a  means  of  removing  the  obstacle. 
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Figure  6b,  A  button  exists  in  the  environment  button  that  will  remove  the  obstacle  and  allow  the  VIP  to  move 
forward.  Team  members  must  search  for  this  button  in  order  to  accomplish  their  goal  of  reaching  the  target 
destination. 
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Figure  6c.  Hostile  agents  are  positioned  throughout  the  environment,  thus  it  is  in  the  team’s  best  interest  to 
coordinate  their  actions  to  avoid  being  outnumbered  by  hostile  agents  and  suffering  heavy  damage.  It  is  also  in  the 
team’s  best  interest  to  move  with  the  VIP  at  all  times  to  protect  him  from  damage. 
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0  .  61136 


Figure  6d.  Once  the  button  (Figure  6b)  has  been  pushed,  the  obstacle  is  removed  and  the  VIP  proceeds  towards  the 
destination.  If  team  members  are  not  with  the  VIP  when  he  begins  to  move,  he  might  encoimter  hostile  agents  and 
take  damage. 
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Figure  6e.  In  each  task,  team  members  eventually  reach  a  target  destination.  When  all  three  team  members  and  the 
VIP  reach  the  destination,  the  task  is  complete. 
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Figure  7a.  In  another  type  of  challenge,  team  members  encounter  a  box  that  can  be  used  to  control  a  ‘train.’  One 
member  must  stay  at  the  control  box  to  control  the  train.  The  optimal  solution  to  the  challenge  is  to  have  one  of 
the  other  members  ride  the  train  to  the  otherwise  inaccessible  parts  of  the  environment,  while  the  third  team  member 
stays  on  the  ground  level  to  operate  the  directional  control  buttons  for  the  train. 
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Figure  7b.  One  team  member  is  riding  the  train,  while  another  waits  below  for  the  rider  to  open  the  gate  blocking 


Figure  Ic.  On  the  ground  floor,  buttons  are  also  present.  These  buttons  adjust  the  direction  of  the  train  to  allow  it 
to  continue  along  the  correct  path.  When  the  directional  button  is  pressed,  an  arrow  shown  on  a  display  visible  to 
the  member  riding  the  train,  changes  directions. 
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Figure  7d.  Buttons  are  located  on  the  walls  that  are  only  accessible  from  the  top  of  the  train.  These  buttons  open 
the  path  for  other  team  members  to  advance  in  the  environment.  However,  the  members  not  riding  the  train  must 
be  sure  to  adjust  the  direction  of  the  train’s  path,  or  the  train  will  be  stuck  in  a  dead-end.  Note  the  arrow  pointing 
left  near  the  top  right  of  the  image.  This  tells  the  team  member  riding  the  train  that  he/she  will  turn  left  unless 
something  happens  to  change  the  direction  of  the  arrow. 
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Figure  7e.  Once  the  team  member  on  the  train  presses  a  button  to  open  a  gate,  a  team  member  on  the  ground  can 
enter  the  next  area  through  the  gate  and  eliminate  any  hostiles  present.  Then,  they  must  press  the  next  directional 
control  button  to  keep  the  train  on  the  correct  path. 


24 


71  68* 


Figure  7f,  If  the  directional  button  on  the  ground  floor  is  pressed  in  time,  the  train  will  continue  along  the  correct 
path  and  allow  the  team  member  on  the  train  to  open  the  next  area  for  the  other  team  members.  Teams  must  work 
together  in  a  coordinated  manner  to  ensure  that  all  the  necessary  events  take  place  in  the  time  allotted.  After 
completing  this  task,  the  team  members  will  reach  a  target  destination  similar  to  the  one  shown  in  Figure  6e. 
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"Coord inative  Ability"  as  a  Function  of  Team  Configuration 


1 

0.8 


0.6 


-0.8 


.1J - - 1 

High,  High  High,  Low  Low,  High  Low,  Low 

Team  Configuration 

Figure  8.  Graph  of  “coordinative  ability”  as  a  function  of  team  configuration.  Scores  displayed  are  the  inverse  of 
the  actual  factor  scores,  such  that  higher  scores  on  this  graph  indicate  better  performance.  That  is,  a  higher  score 
corresponds  to  taking  less  time  to  complete  tasks  and  less  damage  during  the  tasks.  High,  High  refers  to  the  team 
members  being  high  in  cognitive  ability  and  high  in  Social  Orientation.  High,  Low  refers  to  the  team  members 
being  high  in  cognitive  ability  and  low  in  Social  Orientation,  etc.  Error  bars  are  standard  errors. 
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Figure  9.  Illustration  of  the  ‘many-to-one’  model  of  the  final  set  of  regression  analyses. 
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Table  1 


Possible  team  configurations  based  on  cognitive  and  Social  Orientation  ability  measures  in  the  experimental 
design 


Team  Leader 

(Cognitive  /  Social  Orientation) 

Team  Member  1  -  Team  Member  2 

(Cognitive  /  Social  Orientation) 

Number  of  Teams 

to  Complete  Experiment 

HH 

HH-HH 

12  teams 

HH 

HL-HL 

1 1  teams 

HH 

LH-LH 

1 1  teams 

HH 

LL-LL 

10  teams 

Note.  Team  leader  is  always  high  in  both  cognitive  and  social  orientation  abilities.  Other  team  members  are 
categorized  as  high  or  low  in  both  cognitive  and  Social  Orientation  abilities. 
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Table  2 


Dependant  Variables  Measuring  Individual  Performance 


Dependant  Variable 

Individual  Damage  per  Engagement 

Individual  Total  Player  Damage 

Individual  Percent  Ammunition  Used  on  Hostiles 

Individual  Shots  per  Kill 


Individual  Time  per  Kill 


Description 

A  measure  of  the  average  number  of  times  a  team 
member  was  shot  during  encounters  with  hostile  agents 
A  measure  of  the  average  number  of  times  a  team 
member  was  shot  during  a  full  task 
A  measure  of  the  amount  of  ammunition  used  on 
relevant  targets  as  opposed  to  wasted  on  irrelevant 
targets  or  friendly  agents 

A  measure  of  the  average  number  of  bullets  used  to 
eliminate  hostile  agents  encountered  in  the  environment 
(with  careful  aim,  hostile  agents  could  be  eliminated 
with  one  bullet) 

A  measure  of  the  average  amount  of  time  between  a  team 
member  encountering  an  agent  and  the  agent  being 
eliminated 
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Table  3 


Composite  Dependant  Variables  Measuring  Team  Performance 


Dependant  Variable 


Description 


Team  Composite  Damage  per  Engagement 

Team  Composite  Total  Player  Damage 

Team  Composite  Percent  Ammunition  Used  on  Hostiles 

Team  Composite  Shots  per  Kill 
Team  Composite  Time  per  Kill 


The  average  ‘Damage  per  Engagement’  of  the  three  team 
members 

The  average  ‘Total  Player  Damage’  of  the  three  team 
members 

The  average  ‘Percent  Ammunition  Used  on  Hostiles’  of 
the  three  team  members 

The  average  ‘Shots  per  Kill’  of  the  three  team  members 
The  average  ‘Time  per  Kill’  of  the  three  team  members 
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Table  4 


Dependant  Variables  Measuring  Team  Performance 


Dependant  Variable 

Description 

Team  Time  to  Completion 

A  measure  of  the  average  number  of  seconds  required  for 

the  team  to  complete  the  tasks 

Team  VIP  Damage 

A  measure  of  the  number  of  times  the  team  allowed  the 

VIP  to  take  damage  from  hostile  agents 

Team  Percent  Time  VIP  Guarded 

A  measure  of  the  amount  of  time  the  team  kept  at  least 

one  member  within  a  small  distance  of  the  VIP 

Team  Percent  Time  VIP  in  Danger 

A  measure  of  the  amount  of  time  that  hostile  agents  got 

within  a  short  range  of  the  VIP 
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Table  5 


Results  of principal  components  factor  analysis  of  individual  level  dependant  variables. 


Dependant  Variables 

Individual  Level  Factors 

“Combat  Effectiveness” 

“Shooting  Efficiency” 

Individual  Damage  per  Engagement 

0.9f 

-0.03 

Individual  Total  Player  Damage 

0.8f 

O.Of 

Individual  Percent  Ammo  Used  on  Hostiles 

-0.3S 

0.63 

Individual  Shots  per  Kill 

0.03 

0.8f 

Individual  Time  per  Kill 

0.2f 

0.6': 

Note.  Bold  coefficients  indicate  significant  dependant  variable  factor  loadings. 
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Table  6 


Results  of principal  components  factor  analysis  of  team  level  dependant  variables. 


Dependant  Variables 

Team  Level  Factors 

“Coordinative  Ability” 

“VIP  Protection” 

“Shooting  Type” 

Team  Composite  Damage  per  Engagement 

0.91 

0.01 

0.0 

Team  Composite  Total  Player  Damage 

0.94 

-0.0'; 

-0.1 

Team  Composite  Percent  Ammo  Used  on  Hostiles 

-0.3S 

-o.lf 

0.5 

Team  Composite  Shots  per  Kill 

-0.01 

-0.01 

0.8 

Team  Composite  Time  per  Kill 

0.02 

0.3t 

0.5 

Team  Time  to  Completion 

0.81 

o.ot 

-0.0 

Team  VIP  Damage 

O.lf 

0.83 

0.0 

Team  Percent  Time  VIP  Guarded 

0.42 

-0.62 

0.2 

Team  Percent  Time  VIP  in  Danger 

O.K 

0.7S 

0.2 

Note.  Bold  coefficients  indicate  significant  dependant  variable  factor  loadings. 
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Table  7 


Regression  Analysis  Results  for  Individual  Cognitive  and  IPIP  Scores  Predicting  “Combat  Effectiveness” 


Variable 

Unstandardized  Parameter  Standard  Error 

Estimate 

Cognitive  factor 

-0.25 

0.08 

Extraversion 

0.02 

0.01 

Nenroticism 

-0.04 

0.01 

Conscientiousness 

0.02 

0.01 

Agreeableness 

0.03 

0.01 

Openness  to  Experience 

0.01 

0.01 

Note.  Model:  F  (6,125)  =  6.66,  p  <  0.01. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.21. 
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Table  8 


Regression  Analysis  Results  for  Individual  Cognitive  and  ENCAPS  Scores  Predicting  “Shooting  Efficiency  ” 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

-0.14 

0.09 

Social  Orientation 

0.02 

0.01 

Achievement  Motivation 

-0.01 

0.01 

Stress  Tolerance 

0.01 

0.01 

Note.  Model:  F  (4,127)  =  3.55, ;?  <  0.01 . 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.07. 
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Table  9a 


Regression  Analysis  Results  for  Synergy  Model  Cognitive  and  IPIP  Scores  Predicting  **V1P  Protection 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

0.08 

0.23 

Extraversion 

1.3  X  10'* 

1.0  X  10’ 

Neuroticism 

-3.2  X  10^ 

9.0  X  lO"* 

Conscientiousness 

2.0  X  10  ’ 

9.9  X  10-* 

Agreeableness 

-2.0  X  10-’ 

1.0  X  lO  ’ 

Openness  to  Experience 

-2.0  X  10  ’ 

9.5  X  lO"* 

Note.  Model:  F  (6,37) -2.50,;?  <0.05. 
Underlined  type  indicates  p-value  less  than  0.06. 
Adjusted  -  0.17. 
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Table  9b 


Regression  Analysis  Results  for  Synergy  Model  Cognitive  and  ENCAPS  Scores  Predicting  “Shooting  Type" 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

0.30 

0.25 

Social  Orientation 

-7.2  X  10  ’ 

3.0  X  10  ’ 

Achievement  Motivation 

-5.2  X  10  ‘ 

2.3  X  10-* 

Stress  Tolerance 

3.2  X  10  ‘ 

1.3  X  10  * 

Note.  Model:  F  (4,127)  =  3.69, <  0.02. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.20. 
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Table  10a 


Regression  Analysis  Results  for  Weakest-Link  Model  Cognitive  and  IPIP  Scores  Predicting  *'Coordinative  Ability*' 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

-0.56 

0.14 

Extraversion 

0.04 

0.02 

Neuroticism 

-0.06 

0.02 

Conscientiousness 

-7.0  X  lO"* 

0.03 

Agreeableness 

0.02 

0.03 

Openness  to  Experience 

0.04 

0.03 

Note,  Model:  F  (6,37) -4.95,/?  <0.01. 
Bold  type  indicates  p- value  less  than  0.05. 
Adjusted  =  0.36. 
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Table  10b 


Regression  Analysis  Results  for  Weakest-Link  Model  Cognitive  and  IPIP  Scores  Predicting  "VIP  Protection  ” 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

0.14 

0.14 

Extraversion 

0.01 

0.02 

Neuroticism 

-0.01 

0.02 

Conscientiousness 

0.04 

0.03 

Agreeableness 

-0.07 

0.03 

Openness  to  Experience 

-0.10 

0.03 

Note.  Model:  F  (6,37)  =  4.45,/?  <0.01. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.32. 
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Table  10c 


Regression  Analysis  Results  for  Weakest-Link  Model  Cognitive  and  ENCAPS  Scores  Predicting  *'Coordinative 
Ability” 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Enor 

Cognitive  factor 

-0.43 

0.13 

Social  Orientation 

0.01 

0.01 

Achievement  Motivation 

6.2  X  lO"^ 

0.02 

Stress  Tolerance 

-0.04 

0.01 

Note,  Model:  F  (4,39)  =  5.78,/?  <0.01. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.31. 
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Table  lOd 


Regression  Analysis  Results  for  Weakest-Link  Model  Cognitive  and  ENCAPS  Scores  Predicting  "Shooting  Type  " 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

-0.02 

0.13 

Social  Orientation 

-0.01 

0.01 

Achievement  Motivation 

-0.07 

0.03 

Stress  Tolerance 

0.03 

0.01 

Note.  Model:  F  (4,127)  =  A.\9,p  <  0.01. 
Bold  type  indicates  p- value  less  than  0.05. 
Adjusted  =  0.23. 
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Table  11a 


Regression  Analysis  Results  for  Average  Model  Cognitive  and  IPIP  Scores  Predicting  “Coordinative  Ability  ” 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

-0.66 

0.23 

Extraversion 

0.05 

0.03 

Neuroticism 

-0.08 

0.04 

Conscientiousness 

-0.01 

0.04 

Agreeableness 

0.03 

0.05 

Openness  to  Experience 

0.04 

0.05 

Note.  Model:  F  (6,37)  =  3.66,  p  <  0.01. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.27. 
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Table  lib 


Regression  Analysis  Results  for  Average  Model  Cognitive  and  IPIP  Scores  Predicting  "VIP  Protection  ” 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

0.04 

0.25 

Extraversion 

0.04 

0.03 

Neuroticism 

-0.02 

0.04 

Conscientiousness 

0.06 

0.04 

Agreeableness 

-0.10 

0.05 

Openness  to  Experience 

-0.08 

0.05 

Note.  Model:  F  (6,37)  =  2.22,  p  <  0.07. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.15. 
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Table  11c 


Regression  Analysis  Results  for  Average  Model  Cognitive  and  ENCAPS  Scores  Predicting  ''Coordinative  Ability** 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

-0.59 

0.20 

Social  Orientation 

0.02 

0.01 

Achievement  Motivation 

-0.01 

0.03 

Stress  Tolerance 

-0.04 

0.02 

Note.  Model:  F  (4,39)  =  3.67,;?  <  0.01. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.22. 
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Table  lid 


Regression  Analysis  Results  for  Average  Model  Cognitive  and  ENCAPS  Scores  Predicting  “Shooting  Type 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor 

0.01 

0.21 

Social  Orientation 

-0.03 

0.02 

Achievement  Motivation 

-0.07 

0.03 

Stress  Tolerance 

0.05 

0.02 

Note.  Model:  F  (4,39)  =  2.71,;?  <0.05. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.14. 
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Table  12 


Regression  Analysis  Results  for  Individual  Cognitive  and  IPIP  Scores  Predicting  "Coordinative  Ability  " 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Cognitive  factor  (Leader) 

-0.51 

0.39 

Cognitive  factor  (Member  1) 

0.08 

0.18 

Cognitive  factor  (Member  2) 

-0.41 

0.15 

Extraversion  (Leader) 

0.06 

0.02 

Extraversion  (Member  1) 

-0.01 

0.01 

Extraversion  (Member  2) 

0.03 

0.02 

Neuroticism  (Leader) 

-0.03 

0.02 

Neuroticism  (Member  1) 

-0.03 

0.02 

Neuroticism  (Member  2) 

-0.03 

0.02 

Note.  Model:  F  (9,34)  =  4.23,  p  <  0.01. 
Bold  type  indicates  p- value  less  than  0.05. 
Adjusted  =  0.40. 
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Table  13 


Regression  Analysis  Results  for  Individual  Cognitive  and  IPIP  Scores  Predicting  "VIP  Protection  ” 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Extraversion  (Leader) 

0.04 

0.02 

Extraversion  (Member  1) 

0.00 

0.01 

Extraversion  (Member  2) 

-0.04 

0.02 

Neuroticism  (Leader) 

-0.03 

0.02 

Neuroticism  (Member  1) 

0.05 

0.02 

Neuroticism  (Member  2) 

-0.04 

0.02 

Note.  Model:  F  (6,37)  =  4.94,/?  <0.01. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.35. 
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Table  14 


Regression  Analysis  Results  for  Individual  Cognitive  and  ENCAPS  Scores  Predicting  "Shooting  Type  ” 


Variable 

Unstandardized 

Parameter  Estimate 

Standard  Error 

Social  Orientation  (Leader) 

-0.03 

0.02 

Social  Orientation  (Member  1) 

-0.03 

0.01 

Social  Orientation  (Member  2) 

0.02 

0.01 

Achievement  Motivation  (Leader) 

-0.05 

0.02 

Achievement  Motivation  (Member  1) 

-0.02 

0.02 

Achievement  Motivation  (Member  2) 

-0.04 

0.02 

Stress  Tolerance  (Leader) 

0.01 

0.01 

Stress  Tolerance  (Member  1) 

-0.01 

0.01 

Stress  Tolerance  (Member  2) 

0.04 

0.01 

Note.  Model:  F  (9,34)  =  3.37, <  0.01. 
Bold  type  indicates  p-value  less  than  0.05. 
Adjusted  =  0.33. 


48 


Table  15 


Variables  Recorded  by  Data  Logging  Tool 


Variable  Name 


Spawn  Time 


Damage  Taken 


Weapon  Fired 


Button  Pushed 


Door  Opened  /  Closed 


Ammunition  Gained 


Flashlight  /  Tool  Use 


Location 


Entity  Range 


Description 

Records  the  timestamp  of  when  each  team  member  or 
computer  controlled  agent  appears  in  the  environment 
Records  the  timestamp  of  any  time  a  team  member  or 
computer  controlled  agent  is  hit  with  a  weapon 
Records  the  timestamp  of  any  time  a  weapon  is  being 
used 

Records  the  timestamp  and  position  of  any  button 
activated  in  the  environment 
Records  the  timestamp  for  any  doors  or  other  barriers 
opening  or  closing 

Records  the  timestamp  any  time  a  team  member  picks 

up  ammunition  as  well  as  the  amount  picked  up 

Records  the  timestamp  for  any  time  a  flashlight  or  other 

tool  is  used  by  a  team  member 

Records  the  position  of  every  team  member  and 

computer  controlled  agent  in  the  environment  every  0.4 

seconds 

Calculates  and  records  the  distance  of  every  computer 
controlled  agent  from  each  of  the  team  members  every 


0.4  seconds 


